Systematic Analysis of Engineering Change Request Data - Applying Data Mining Tools to Gain New Fact-Based Insights

Large, complex system development projects take several years to execute. Such projects involve hundreds of engineers who develop thousands of parts and millions of lines of code. During the course of a project, many design decisions often need to be changed due to the emergence of new information. These changes are often well documented in databases, but due to the complexity of the data, few companies analyze engineering change requests (ECRs) in a comprehensive and structured fashion. ECRs are important in the product development process to enhance a product. The opportunity at hand is that vast amount of data on industrial changes are captured and stored, yet the present challenge is to systematically retrieve and use them in a purposeful way.This PhD thesis explores the growing need of product developers for data expertise and analysis. Product developers increasingly refer to analytics for improvement opportunities for business processes and products. For this reason, we examined the three components necessary to perform data mining and data analytics: exploring and collecting ECR data, collecting domain knowledge for ECR information needs, and applying mathematical tools for solution design and implementation.Results from extensive interviews generated a list of engineering information needs related to ECRs. When preparing for data mining, it is crucial to understand how the end user or the domain expert will and wants to use the extractable information. Results also show industrial case studies where complex product development processes are modeled using the Markov chain Design Structure Matrix to analyze and compare ECR sequences in four projects. In addition, the study investigates how advanced searches based on natural language processing techniques and clustering within engineering databases can help identify related content in documents. This can help product developers conduct better pre-studies as they can now evaluate a short list of the most relevant historical documents that might contain valuable knowledge.The main contribution is an application of data mining algorithms to a novel industrial domain. The state of the art is more up for the algorithms themselves. These proposed procedures and methods were evaluated using industrial data to show patterns for process improvements and cluster similar information. New information derived with data mining and analytics can help product developers make better decisions for new designs or re-designs of processes and products to ensure robust and superior products

Applying Design Analytics to Understand Engineering Change Request Information

Large complex system development projects take several years to carry out. Such projects involve hundreds of engineers who develop tens of thousands of parts and millions of lines of code. During the course of a project, many design decisions often need to be changed due to the emergence of new information. These changes are often well documented in databases but, due to the complexity of the data, few companies analyze engineering change requests (ECRs) in a comprehensive and structured fashion. ECRs are important and plentiful in the product development process in order to enhance a product.This thesis sets out to explore the growing need of product developers for data expertise and analysis. Product developers are increasingly looking towards analytics for improvement opportunities within business processes and products. For this reason, we look at the three components necessary to perform data mining and data analytics: exploring and collecting ECR data, collecting domain knowledge towards ECR information needs and applying mathematical tools for solution design and implementation.Results show two software tools including visuals of ECR text mining and design structure matrix. The tools were evaluated using industrial data showing patterns and improvement for products and process. Results also show a list of engineering information needs towards ECRs. New information derived with data mining and analytics can thus support product developers in making better decisions for new designs/re-designs of processes and products that lead to robust and superior products

Modeling industrial engineering change processes using the design structure matrix for sequence analysis: a comparison of multiple projects

The problem at hand is that vast amount of data on industrial changes is captured and stored; yet the present challenge is to systematically retrieve and use them in a purposeful way. This paper presents an industrial case study where complex product development processes are modeled using the design structure matrix (DSM) to analyze engineering change requests sequences. Engineering change requests are documents used to initiate a change process to enhance a product. Due to the amount of changes made in different projects, engineers want to be able to analyze these change processes to identify patterns and propose the best practices. The previous work has not specifically explored modeling engineering change requests in a DSM to holistically analyze sequences. This case study analyzes engineering change request sequences from four recent industrial product development projects and compares patterns among them. In the end, this research can help to identify and guide process improvement work within projects

Natural language processing methods for knowledge management - Applying document clustering for fast search and grouping of engineering documents

Product development companies collect data in form of Engineering Change Requests for logged design issues, tests, and product iterations. These documents are rich in unstructured data (e.g. free text). Previous research affirms that product developers find that current IT systems lack capabilities to accurately retrieve relevant documents with unstructured data. In this research, we demonstrate a method using Natural Language Processing and document clustering algorithms to find structurally or contextually related documents from databases containing Engineering Change Request documents. The aim is to radically decrease the time needed to effectively search for related engineering documents, organize search results, and create labeled clusters from these documents by utilizing Natural Language Processing algorithms. A domain knowledge expert at the case company evaluated the results and confirmed that the algorithms we applied managed to find relevant document clusters given the queries tested

Analysis of Engineering Change Requests Using Markov Chains

Engineering change requests are important and plentiful in the product development process to enhance\ua0a product. In this paper we use Markov chains on ECRs in a large product development project and\ua0display the results in a Markov chain DSM. The DSM shows statistical probability of a transition\ua0pathway for an industrial design process and together with engineering domain knowledge we identify\ua0patterns and improvement opportunities. It turns out that 8% of ECRs are closed directly after creation,\ua0most common pathway is not followed in early statues and status iterations are seen in the DSM

Analysis of Engineering Change Requests Using Markov Chains

Engineering change requests are important and plentiful in the product development process to enhance\ua0a product. In this paper we use Markov chains on ECRs in a large product development project and\ua0display the results in a Markov chain DSM. The DSM shows statistical probability of a transition\ua0pathway for an industrial design process and together with engineering domain knowledge we identify\ua0patterns and improvement opportunities. It turns out that 8% of ECRs are closed directly after creation,\ua0most common pathway is not followed in early statues and status iterations are seen in the DSM

Supporting Knowledge Re-Use with Effective Searches of Related Engineering Documents - A Comparison of Search Engine and Natural Language-Based Processing Algorithms

Product development companies are collecting data in form of Engineering Change Requests for logged design issues and Design Guidelines to accumulate best practices. These documents are rich in unstructured data (e.g., free text) and previous research has pointed out that product developers find current it systems lacking capabilities to accurately retrieve relevant documents with unstructured data. In this research we compare the performance of Search Engine & Natural Language Processing algorithms in order to find fast related documents from two databases with Engineering Change Request and Design Guideline documents. The aim is to turn hours of manual documents searching into seconds by utilizing such algorithms to effectively search for related engineering documents and rank them in order of significance. Domain knowledge experts evaluated the results and it \ua0shows that the models applied managed to find relevant documents with up to 90% accuracy of the cases tested. But accuracy varies based on selected algorithm and length of query

Towards Big-Data Analysis of Deviation and Error Reports in Product Development Projects

Large complex system development projects, such as complete truck development projects, take several years to carry out. They involve hundreds of engineers who develop tens of thousands of parts and millions of lines of codes. During a project, many design decisions often need to be changed due to emergence of new information. The bulk of these changes are requested late in the development process. It is known that changes late in the development process are very costly and run a risk of delaying the project. These changes are often well documented in databases, but, due to the complexity of the data, few companies analyze engineering change in a comprehensive and structured fashion. This paper argues that “big data” (specifically data mining) analysis tools can be applied for such analyses and proposes a process for carrying out the analysis and using the results for product and development process improvement. The paper further accounts for experiences gained from testing the approach on a dataset consisting of 4,000 deviation and error reports that were created during a truck development project

Design Analytics is the Answer, But What Questions Would Product Developers Like to Have Answered?

There is a growing need for data expertise and data analysis. Companies are looking more towards analytics for improvement opportunities within the business and products. Data collection is growing at a fast pace and we need capabilities to be able to analyze it. The data volume that companies are sitting on makes this task even more important. The paper presents interviews performed with product developers who have worked on a large complex system development project. The findings explain questions and needs developers are facing and what answers they are looking for with data mining. By Identifying beneficial and meaningful outputs from data mining and data analytics, developers can be supported in making better decisions for a new designs/re-designs and ultimately make a superior robust product. The paper further accounts for 20 heterogeneous purposefully sample interviews, ranging in project roles from product development to manufacturing and testing

Trends, observations and drivers for change in systems engineering design

Manufactures, developing products, need to adapt and improve their practices taking advantage of technology advancements and simultaneously develop products and solutions to fit a new world. This paper discusses how societal and technological trends drive the need for change and evolution in what is called Systems Engineering Design (SED), indicating a systems view on engineering design. Through an analysis and selected examples it is argued that SED capabilities need to better address the width and complexity of design problem, takes advantages of increased computational power and sensing technologies to master future challenges. An important factor for successful deployment and change in industrial context, is the need for interactive and visual aids and easily accessible support methods. This can pave the way also for advanced SED suppor